1. Field of the Invention
This invention relates to storage area network data centers. Particularly, this invention relates to optimizing placement of applications within a heterogeneous storage area network data center where each application needs a certain amount of CPU resource and a certain amount of storage resource.
2. Description of the Related Art
Server virtualization is becoming an important data center technology. Multiple virtual servers can now be run on a single physical server, using tools like VMware and Xen, and can also be moved seamlessly across physical servers. This consolidates hardware and improves utilization. However, managing such environments remains a challenge.
Furthermore, nodes in the Storage Area Network (SAN) are becoming increasingly heterogeneous. Switches and storage controllers, for example, now have non-traditional availability of processing power. With virtualization technologies, proximity of these nodes from storage resources can yield significant performance benefits by careful placement of applications. This automatic dynamic management of where to run each application is a challenging task that needs to be addressed for realizing utility computing.
Current products in this space, like VMware Infrastructure-3, continuously monitor CPU resource utilization in a data center, and based on certain policies can prioritize how CPU resources are allocated to application virtual machines. However, it only concentrates on optimization of CPU resources for applications, staying oblivious of the underlying storage system. On the other end, tools like Ergastulum, Hippodrome and Minerva, attempt to find suitable storage system configurations taking into account only the storage requirements of applications. There does not exist any framework that attempts to couple an applications' storage and CPU requirements for optimized placement decisions.
Such coupling of applications' storage and CPU placement becomes important as the data centers become large and the variability in the SAN nodes and fabric grows. The hardware and SAN fabric in data center is incrementally built over time, so older less powerful components coexist with newer more powerful ones. Coupled placement of CPU and storage also becomes essential if one were to take advantage of the specialized processing power available at modern fabric switches and storage controllers. Recent hardware trends have introduced this non-traditional availability of processing power. For example, Cisco MDS 9000 switches have multiple blades with Intel x86 processors that can be used to run various applications. On a parallel, IBM DS8000 storage controllers have POWER5 processors that have recently been used to run DB2 database system. Previously, these resources were used only for specialized applications, but now with virtualization technologies these nodes can also be added to the resource pools.
Many applications can drastically improve performance by running on these specialized nodes. For example, a database application that requires lengthy table scans can achieve higher throughput and lower response times by running on the storage controller that hosts its database. The proximity of these processing resources to the storage resources makes them preferable locations for applications that access storage frequently. An independent placement of CPU and storage fails to capture this preference for suitable applications.
Such problems arise in situations where two distinct item types need to be allocated to two resources and the cost of the allocation is determined by the choice of those resources. For example, for placing CPU and storage in a storage area network, depending upon where application storage is placed, all CPU nodes have a certain “affinity” to that storage node and performance of that application will depend on where the CPU is placed. Similar questions arise in deciding where to produce and where to process items in manufacturing. These questions also arise in grid computing and other domains where data and computational resources need to be placed in a coupled manner.
Most current solutions look at only placing one item in a set of resources, e.g., File Allocation Problem, Generalized Assignment Problem and many area specific approximation algorithms, e.g., storage placement, CPU load balancing etc.
This problem captures the basic questions inherent in placing two items in a coupled manner. The NP-Hard nature of the problem can be established by reducing to the 0/1 Knapsack problem. Even if a simpler case of this problem, involving two exclusive resource-A nodes (one is a catch-all node of infinite capacity and large cost) and fixed resource-B allocation, can be solved, it can be used to solve the knapsack problem. This can be solved by making the second resource-A node correspond to the knapsack and setting the costs and Areq requirements accordingly. Having to decide coupled placements for both A and B with general cost functions makes the problem more complex.
If either Resource-A or Resource-B allocations are fixed and only the other needs to be determined then there is related work in the computing domain for storage or CPU placement of applications in a data center: File Allocation Problem placing files/storage assuming CPU is fixed; Minerva, Hippodrome assume CPU locations are fixed while planning storage placement; Generalized Assignment problems: Assigning tasks to processors (CPUs). They have been well-studied with several heuristics proposed but they do not consider the coupled allocation.
If the resource requirement (for A and B) for items can always be split across multiple resource nodes, then one could also model it as a multi-commodity flow problem, one commodity per item, introduce a source node for the resource-A requirement of each item and a sink node for resource-B requirement, with the source node connected to all nodes with nonzero A capacity, nonzero resource-B nodes connected to the sink node and appropriate costs on the resource-A and resource-B node pairs. However multi-commodity flow problems are known to be very hard to solve in practice even for medium sized instances. And if the splitting is not justifiable for items (e.g., it requires sequential processing at a single location), then we would need an unsplittable flow version for multi-commodity flows, which becomes even harder in practice.
Another important aspect of the problem is non-uniform costs. If the cost for each item Ii were the same for all (vj, vk) pairs then the problem could be simplified to placement for resource-A and B independently without coupling.
U.S. Patent Application Publication No. 2003/0051021 by Hirschfield et al., published on Mar. 13, 2003, discloses a virtualized logical server cloud that enables logical servers to exist independent of physical servers that instantiate the logical servers. Servers are treated as logical resources in order to create a logical server cloud. The logical attributes of a logical server are non-deterministically allocated to physical resources creating a cloud of logical servers over the physical servers. Logical separation is facilitated by the addition of a server cloud manager, which is an automated multi-server management layer. Each logical server has persistent attributes that establish its identity. Each physical server includes or is coupled to physical resources including a network resource, a data storage resource and a processor resource. At least one physical server executes virtualization software that virtualizes physical resources for logical servers. The server cloud manager maintains status and instance information for the logical servers including persistent and non-persistent attributes that link each logical server with a physical server.
U.S. Patent Application Publication No. 2003/0033487 by Pfister et al., published on Feb. 13, 2003, discloses a method, apparatus, and computer implemented instructions for managing a plurality of caches of data, wherein the data processing system includes a plurality of independent computers. In response to initiating a read operation to read data on a data block, an indication is posted on a directory of data blocks identifying the computer that now holds a copy of that block and a location in the memory of that computer where a flag associated with that block is held. Then in response to initiating a write operation on that data block, messages are sent to all the computers holding that block which resets the said flag, thus informing each computer that the data in that block is no longer valid. These messages are sent using means that perform that flag reset without, in the preferred embodiment, any overhead of interruption of processing on the computers where the flags reside.
U.S. Pat. No. 7,003,688 by Pittelkow et al., issued on Feb. 21, 2006, discloses that a fibre channel storage area network (SAN) provides virtualized storage space for a number of servers to a number of virtual disks implemented on various virtual redundant array of inexpensive disks (RAID) devices striped across a plurality of physical disk drives. The SAN includes plural controllers and communication paths to allow for fail-safe and fail-over operation. The plural controllers can be loosely-coupled to provide n-way redundancy and have more than one independent channel for communicating with one another. In the event of a failure involving a controller or controller interface, the virtual disks that are accessed via the affected interfaces are re-mapped to another interface in order to continue to provide high data availability. In particular, a common memory storage device is connected to the back-ends of every controller to provide a storage area. In this manner, the common memory storage device can be accessed via operations similar to those a controller already uses to presently access the physical disks which are connected to the back-end of the controllers.
U.S. Pat. No. 6,799,258 by Linde, issued on Sep. 28, 2004, discloses methods and apparatus for point-in-time volumes are provided. A relationship is enabled between a source volume and point-in-time volume. Copying a data chunk to the point-in-time volume before a write operation modifies the data chunk on the source volume dynamically creates the point-in-time volume. The point-in-time volume can be accessed in read/write mode as a general purpose data storage volume. Other embodiments comprising additional features, such as a forced migration process, are also provided.
Existing systems and methods do not address the problem of planning application placements in virtualized heterogeneous SAN data centers. Thus, there is a need in the art for effectively planning placement of applications in data centers when each application needs a certain amount of CPU resource and a certain amount of storage resource. Particularly, there is a need for programs, methods and systems that place applications having distinct processing and storage requirements considering both in an integrated manner. There is a need for such programs, methods and systems to operate with storage area networks having resource nodes that are connected and each provide different sets of processing and/or storage resources. These and other needs are met by the present invention as detailed hereafter.